Effects of Reflective Thinking in the Process of Designing Software on Students’ Learning Performances

The purpose of this study is to examine the effects of reflective thinking effects in the process of designing software on students’ learning performances. The study contends that reflective thinking is a useful teaching strategy to improve learning performance among lower achieving students. Participants were students from two groups: Higher achieving students were the control group, and lower achieving students were the experimental group. The experimental group students wrote weekly diaries demonstrating reflective thinking in the processes of designing software. The study’s results show the evaluation scores of the two groups of students’ software designs were comparable. The abilities of the experimental group to comment on a design improved. This shows that a reflective thinking strategy did have positive effect on improving lower achieving students’ learning performances, especially in the process of software design. Discussion of the results and suggestions for future study appear at the end of this study. INTRODUCTION Currently, numerous software design tools are helpful for rapidly constructing an environment of digital learning, which contributes to achieving expected learning goals and efficacy. However, when considering learners’ backgrounds, software developers should ponder principles of accessibility and usability, to integrate learning theories into designs (Hackett & Parmanto, 2006; Hsieh, 2008; Kidney, Cummings & Boehm, 2007; Lam, Lam, Lam & McNaught, 2009; Powell & Murphy, 2009; Zhang, Zhang, Duan, Fu & Wang, 2010). For example, when a web designer develops a search engine, an invitation to users allows participation and exploration of behavior and cognition simultaneously, and application of a learning theory should be used to clarify issues of usability and accessibility in web design. Subsequently, the web designer can revise the original design’s concepts and finalize the search engine’s design. According to Dewey (1933) reflective thinking is, “the kind of thinking that consists of turning a subject over in the mind and giving it serious and consecutive consideration” (p. 3). The software design process requires reflective thinking, in addition to ordinary thinking; namely, previous thoughts and conduct should be reviewed through “reflective thinking,” prepared for deep learning (Dewey, 1933). If time for reflective thinking is available during the process of software design, capitalizing on opportunities for undertaking reflective thinking involves acquiring descriptive content (or facts), procedural knowledge (conceptual or induced from formulas), and high-level learning (i.e., deep learning). An example of reflective thinking process is: Teachers encourage students demonstrating low learning performance to access previous learning, revisit techniques employed to gain the learning, and estimate knowledge deficits requiring fulfillment. This reflective thinking process has the potential to improve performance and unique personal creativity of low learning students (Schunk, 2004). According to Social Cognitive Theory, social environments and personal beliefs or cognition affect the ways humans thinking. In other words, “self-efficacy” (i.e., an individual's self-confidence level for completing a special task) will be different, depending on different situations (Bandura, 2002). Some studies’ empirical results indicated that students having higher levels of self-efficacy achieved higher learning performance, and vice versa. Since students gained confidence from positive experiences of achieving high learning performance, their high levels of self-efficacy further advance (McMurray & Sanft, 2005; Phan, 2007, 2008). In addition, a few studies examined whether or not students involved in reflective learning processes attained significant benefit in learning performance, and their reflective learning outcomes influenced their levels of self-efficacy. Therefore, the current research designs an experimental study to verify whether or not reflective thinking effects exist for students in a software design course, especially those demonstrating lower achievement. Overall, this study’s three core questions are: Do lower achieving students’ learning performances improve from reflective thinking processes? Do lower achieving students’ self-efficacies for computer use enhance due to the effects of reflective thinking? Finally, through exploration what reflective thoughts are produced and then discovered from lower achieving students’ perspectives for a commenting task? TOJET: The Turkish Online Journal of Educational Technology – April 2012, volume 11 Issue 2 Copyright  The Turkish Online Journal of Educational Technology 89 LITERATURE REVIEW Clarification of this study’s purpose requires discussion of the meaning of learning through reflective thinking and the causal relationship between reflective thinking outcomes and individuals’ levels of self-efficacy. The literature review also includes the design principles and the multi-dimensional assessment criteria adopted in the software design course for implementing learning through reflective thinking. Learning through Reflective Thinking The conceptual definition of the term, reflective thinking, originates with a proposal from the scholar John Lock in 1690, and subsequently the scholar John Dewey transformed this concept into an operational principle, which asserts that reflective thinking can result in true, purposeful, and meaningful learning (Dewey, 1933). The definition of reflective thinking suggests that the process can result in great value, despite the effort required. The process of reflective thinking affords personal creativity, meaning, and criticism from learning activities. In terms of exploration of reflective learning, the process of learning encompasses deep or high-level learning, which means engaging in critical thinking, obtaining cognitive and metacognitive awarenesses, operating with sophisticated conceptual thinking, and originating creative ideas to solve problems (Chang & Chou, 2011; Hatton & Smith, 1995; McCrindle & Christensen, 1995; Thorpe, 2004). In other words, reflective learning transforms and re-digests acquired knowledge to solve problems and demonstrate personal creativity – perhaps, highlighting personal values during knowledge acquisition and integration. Prior study results indicated the positive impact of reflective thinking on learning achievement (Ersozlu & Arslan, 2009; McCrindle & Christensen, 1995; Sheorey & Mokhtari, 2001). Those results showed either significantly improved scores obtained by experimental groups or obviously comparable learning performance in the assigned learning tasks when compared other students in control groups. In addition, as an individual learner proceeds through a reflective thinking process, awareness of strategies for learning may simultaneously enhance. The learner’s perception of level of ability to learn (i.e., self-efficacy) increases as well. The McMurray and Sanft (2005) study also indicated a strong correlation between reflective thinking and self-efficacy. The Phan (2007) study empirically verified the causal effects of reflective thinking and self-efficacy on academic performance. The importance is the necessity of encouraging students to think reflectively during learning processes. At present, one of the most commonly seen avenues for learning through reflective thinking in classrooms is practice through projects. Students, individually or in groups, can learn from solving case problems, learn from doing, and do from learning. Also, students can actively participate in other activities before, during and after the class (e.g., preview, review, and discuss learning content), to enhance the content quality of reflective thinking and then achieve the expected learning results (Thomas, 2000). As for other ways of learning through reflective thinking, the literature suggests that teachers adopt different teaching strategies: completing reflection sheets, writing a reflections diary, storytelling, or debating openly (Chen, Kinshuk, Wei & Liu, 2011; Hatton & Smith, 1995; McKillop, 2005). The current study asks students to write reflection diaries, and in deference to today’s high technology, pen and paper are not necessarily the only technique for recording. Instead, student wrote personal reflections of learning on blogs (Efimova & Fiedler, 2004) and had the freedom to add representative images and pictures to supplement their reflections. Self-Efficacy Based on the perspective of Social Cognitive Theory, “self-efficacy,” proposed by Bandura is a learner’s cognition of self-confidence and the capability to achieve a degree of competence in a specialty (Bandura, 2008). It is an individual’s evaluation of self-confidence and belief in ability to accomplish a mission (Schunk, 2004). Self-efficacy is different from ability. With expected outcomes, self-efficacy is a person’s cognition of ability when taking action. Cognition changes, as exposure and retention of information expand (Bandura, 1986). For example factors, such as personal emotional change, anticipation for results, awareness of others’ expectations, previous experience, environmental conditions, and so on, can affect an individual’s self-evaluation of ability. Among the factors, arguably, previous experience is a strong predictor of self-efficacy, because previous positive or negative learning experience influences the extent of a learner’s evaluation of self-efficacy. Those who had positive learning experiences gain enhanced self-efficacy more easily than those who had negative learning experiences. In addition, a learner’s self-efficacy will likely change during different times, and situations, and build inner value for cognition for self-ability (Bandura, 2002; İşman & Çelikli, 2009; Topkaya, 2010). For more than 20 years after 1977, much research of self-efficacy appeared, including correlations, pre-test and post-test comparisons, and experimental studies. These studies mainly explore the influence of self-efficacy on learning achievement, including: solving conceptual math problems, writing, and reading abilities, learning TOJET: The Turkish Online Journal of Educational Technology – April 2012, volume 11 Issue 2 Copyright  The Turkish Online Journal of Educational Technology 90 motivation (Bong & Clark, 1999; Schunk, 1991; Zimmerman & Bandura, 1994; Zimmerman, Bandura & Martinez-Pons, 1992), occupation choice or career decision, etc. (Betz & Hackett, 1981; Chaney, Hamoond, Betz & Multon, 2007; Hartman & Betz, 2007). The literatures, suggests that self-efficacy development encompasses many patterns. Self-efficacy mentioned in various studies circumscribes the same concept, but within differing contexts (Multon, Brown & Lent, 1991; Pajares & Miller, 1994; Schunk, 1991; Zimmerman, 1995; Zimmerman & Bandura, 1994; Zimmerman, Bandura & Martinez-Pons, 1992). For example, self-efficacy in schoolwork is the student’s cognition of ability for achieving goals in learning. Self-efficacy in teaching is a teacher’s cognition of an ability to help students learn, and consequently influences motivation and willingness to prepare instructional materials and activities. Self-efficacy in computer technology is a self-rating ability to apply computers to diverse situations, such as searching information via the Internet and using multi-media software. Consequently, questionnaires developed for different studies for measuring self-efficacy in various contexts are unique (Hsieh, 2009). For measuring self-efficacy of computer use, the General Self-Efficacy Scale (GSE), developed by Schwarzer and Jerusalen, is the most common questionnaire, which is able to predict a student’s performance with computers in daily life, under pressure, and so on (Schwarzer & Jerusalem, 1995). However, according to studies conducted by Joo, Bong and Choi, a questionnaire’s design, which has high credibility for measuring self-efficacy for Internet use merely measures a student’s ability to search web pages effectively, not a more general ability for learning (Joo, Bong & Choi, 2000). Thus, instead of utilizing GSE, the Thatcher and Perrewe (2002) Computer Self-Efficacy Questionnaire is an alternative for determining students’ abilities to search web pages. However, to measure students’ abilities of packaged software using, this questionnaire maybe not an appropriate one. This study adopts the Compeau and Higgins (1995) Computer Self-Efficacy Questionnaire, researched and designed by Compeau and Higgins (1995). The questionnaire contains ten Yes/No questions and measures both the magnitude of self-efficacy, so it can provide data of students’ abilities to use packaged software. Compeau and Higgins established the reliability and validity of analyses obtained from this questionnaire after using Partial Least Squares to test a research model. The results of their analyses showed the questionnaire’s high internal consistency (reliability coefficient greater than 0.8), and strong construct validity (higher than 0.7). Thus, to examine whether or not lower achieving students, who had undertaken learning through reflective thinking, would demonstrate significant change in self-efficacy, after completing an animation software design assignment, adoption of this questionnaire is more useful and appropriate than others. Design Principles and Multi-Dimensional Assessment Criteria Arguably, design principles, such as accessibility and usability, are fundamental skills for novice designers. Other design principles, such as aesthetics, attention-getting, friendly, responsiveness, simplicity, error tolerance and reliability, are also software designers’ considerations. Also, design principle may have application in some, but not all, design projects. For example, novice web designers must execute several accessibility checks, including HTML elements, multimedia elements, web tools, and advanced scripting (Institute for Interactive Technology, 2006), to ensure the accuracy of the information presented on each web page; usability checks are likely necessary as well. For novice software designers, prior to the processes of software design, all usability checks are important considerations, including: compatibility, consistency or coherence, directness (WYSIWYG-what you see is what you get), interactivity, and user-in-control (Nielsen, 1994). Expert software designers intuitively consider most design principles in their software development processes, without consciously applying design principles, they are aware of most users’ preferences and update designs to comply with the latest trends. The most effective way to achieve most design principles’ requirements is to invite software users entering the processes of software design to test software beta versions, even though such testing is sometimes very time consuming (Ropinski, Meyer-Spradow, Steinicke & Hinrichs, 2006). Then software designers and users have chance to communicate with each other during development before releasing the software to the market. This approach, called user-oriented design, is increasingly popular among both novice and expert software designers. Overall, by taking design principles into consideration or taking user-oriented design approach, designers have gone through a reflective thinking process during the process of design. In order to objectively assess students’ performances for designing animation software, this study adopts multi-dimensional assessment criteria provided by the Association to Advance Collegiate Schools of Business (AACSB, http://www.aacsb.edu/). In recent years, AACSB has supervised many educational institutions to establish multi-dimensional assessment criteria to ensure quality of global business education not only in the United States, but also around the world. The academies achieving the criteria’s quality standards gain certification from AACSB (http://www.aacsb.edu/accreditation/). This study evaluates students’ learning TOJET: The Turkish Online Journal of Educational Technology – April 2012, volume 11 Issue 2 Copyright  The Turkish Online Journal of Educational Technology 91 performances according the AACSB’s five criteria developed for a software design course: Information Technology (IT), Oral Communication (OC), Problem Solving (PS), Value and Professionalism (VP), and Creativity and Innovation (CI). The contents of each assessment criterion are: IT—including four sub-criteria: A student is proficient in instructions and functions of the animation design software, “Adobe® Flash CS3,” is able to integrate other Adobe® software that allows editing web pages, pictures, videos, and so on, can configure a design to appear in its entirety on a personal website, accepts the Internet as an important tool for accessing information pertinent to design. OC—including two sub-criteria: A student is comfortable using any mode or media of communication for delineating software design ideas, especially, demonstrating skill with technology for effective communication. PS—including three sub-criteria: A student can correctly debug the software from error codes that appear during the process of software design (the most important one), is able to conduct immediate tests on the results of different animation software designs, can revise developed software according to principles of design. VP—including two sub-criteria: A student can attend class on time as well as demonstrate an active attitude toward the process of learning (i.e., maintain an interest in software design), and is willingly undertakes software development according to the principles of design. CI—including two sub-criteria: A student can use different methods to solve problems arising in the process of software design, complete a design that is instinctively interesting to users. RESEARCH METHODS The research methods implemented in this study include: Study’s framework, experimental design, including study procedures, participants, and data collection, and analysis methods. Study Framework Reflective thinking is an important strategy in students’ learning processes. According to the literature, for enhancing problem-solving abilities as well as increasing creativity, students should receive frequent encouragement to think reflectively when engaging in the process of learning or designing software. Thus, this study examines the effects of reflective thinking on students’ learning performances during software design processes. The assumption is that for software design assignments, lower achieving students will demonstrate learning-process improvement after thinking reflectively and achieve performance comparable to higher achieving students. A further assumption is that all students’ computer self-efficacy significantly improves after acquiring skills in a software design course. After practicing reflective thinking in the process of design, especially, among those deemed to have lower achieving learning performances at the beginning of the software design course, self-efficacy for computer use should be equivalent to those students demonstrating higher achievement. This study adopts the Compeau and Higgins (1995) questionnaire, which has 10 questions answerable by “YES” or “NO.” If the answer is the former, the students can make a personal self-assessment of strength of self-confidence on a 1 to 10 scale. All the students completed the questionnaire twice, one before the beginning of the course and again at the course’s completion. The operatational assumption is that students demonstrating low performance in the software design course could acquire abilities for software design and increased their levels of self-efficacy for computers by thinking reflectively. Figure 1 illustrates this study’s framework. Two proposals for null hypotheses are: Null Hypothesis 1 – A reflective thinking strategy cannot aid lower achieving students to reach learning performance comparable with higher achieving students in a software design course. Null Hypothesis 2 The level of computer self-efficacy after the course has no significant incensement compared with that before the course for students in the lower achieving group. Figure 1: Study framework. Reflective thinking’s effects Learning performance Computer self-efficacy H1